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~y  for  strict  sense  optimal  controls,  then  standard  methods  are  not 
adequate  to  prove  it.  There  is  a  similar  difficult  in  proving 
existence  of  optimal  controls  with  complete  observations  with 
singular  noise  coefficient,  if  the  term  "complete  observations"  is 
taken  in  the  strict  sense  that  depends  on  the  past  of  the  Wiener 
process  driving  the  system. 


EXISTENCE  OF  OPTIMAL  CONTROLS  FOR 
PARTIALLY  OBSERVED  DIFFUSIONS 


% 


WENDELL  H.  FLEMING  AND  ETIENNE  PARDOUX 


1.  Introduction .  In  this  paper  we  are  concerned  with  the  existence 
of  optimal  controls  for  problems  of  the  following  kind.  Let  Xt 
denote  the  process  which  we  wish  to  control,  Y  the  observation 
process,  and  the  control  process,  0  <  t  <  T,  with  T  fixed 

The  state  and  observation  processes  are  governed  by  stochastic 
differential  equations 

(a)  dXt  =  b(Xt,Yt,Ut)dt  +  c(Xt,Yt)dWt 

(1.1) 

(b)  dYt  =  h(Xt)dt  +  dW  . 

X  has  values  in  N-dimensional  ]R^,  Y  values  in  IR^,  and  U 

L  X.  L 

values  in  ^  c  ]RL .  XQ  has  given  distribution  U,  and  YQ  =  0. 

In  (1.1),  W  and  W  are  independent  standard  Wiener  processes,  with 
values  in  IR^dR^  respectively.  The  matrix  o  is  thus  N  *  D. 

The  problem  is  to  minimize  a  criterion  of  the  form 

rT 

(1.2)  J  =  E{J^F(Xt,Ut)dt  +  G(Xt)}. 


It  is  customary  to  require  that  be  measurable  with  respect  to 

the  o-algebra  generated  by  the,  observations  Y  ,  0  <  s  <  t.  We 
call  this  the  strict  sense  version  of  the  problem  (§6).  For 
several  years  the  question  of  proving  a  general  theorem  about  existence 
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of  optimal  controls  in  the  strict  sense  has  been  open.  We  do  not 
obtain  such  a  result  here.  In  fact,  our  results  together  with  a 
counterexample  of  Varadhan  (§6)  strongly  suggest  that,  if  there  is 
indeed  a  general  existence  theorem  for  strict  sense  optimal  controls, 
then  standard  methods  are  not  adequate  to  prove  it.  There  is  a 
similar  difficulty  in  proving  existence  of  optimal  controls  with 
complete  observations  with  singular  noise  coefficient  o,  if  the 
term  "complete  observations"  is  taken  in  the  strict  sense  that  U^. 
depends  on  the  past  of  the  Wiener  process  driving  the  system. 

Instead  of  allowing  only  strict-sense  controls,  we  obtain 
existence  of  a  minimum  in  a  wider  class  of  controls.  Roughly  speaking, 
this  wider  class  is  obtained  as  follows.  Let 

(1.3)  Z  =  expIphCX  VdY  -  i  ft|h(X  )  1 2ds] . 

t  JQ  s  s  i  J0  s 

Then  are  independent  standard  Wiener  processes  under  a  new 

O 

probability  measure  P  related  to  the  original  probability  measure 

O 

°  dP  -1 

P  by  3p  '  Zt  ■  In  the  wider  sense  formulation  we  wish  to  require 
merely  that  be  independent  of  future  increments  Yr  -  for 

O 

t  <  r  and  independent  of  the  W  process,  with  respect  to  p.  In 
§2  we  give  a  precise  formulation  of  this  idea,  in  which  the  control 
is  defined  as  the  joint  probability  distribution  measure  w  of 
the  processes  Y,U. 

Our  method  depends  on  introducing  a  "pathwise"  version 
Y  U 

At  =  ht’  of  an  unnormalized  conditional  distribution  measure  for 
Xt  given  past  values  of  the  observation  process  Y  and  the  control 
process  If  (§3).  An  important  fact  is  the  continuous  dependence 
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of  A  on  Y,U  and  the  initial  distribution  u  (Lemma  3.2).  In 
§  4  we  introduce  a  "separated"  control  problem,  equivalent  to  the 
original  problem  formulated  in  §2.  The  "state"  in  the  separated 
problem  is  the  measure  A^.  In  §4  we  study  the  dynamics  of  A^, 
using  a  method  of  forward  and  backward  partial  differential  equations. 
Similar  ideas  were  used  in  [13],  for  the  nonlinear  filter  problem. 

The  forward  equation  (5.4)  is  linear  parabolic  (possibly  degenerate) 
with  coefficients  depending  parametrically  on  observations  Y  and 
controls  U^.  Under  suitable  regularity  assumptions,  A^  has  a 
density  q(t,x),  related  in  a  simple  way  to  a  solution  p(t,x)  of 
the  forward  equation  via  (5.6).  Without  the  regularity  assumptions, 
one  uses  instead  a  weak  sense  version  (5.4')  of  the  forward  equation. 

We  also  show  that  A  satisfies  the  Zakai  equation  (5.8)  of  non¬ 
linear  filtering.  The  method  of  backward  and  forward  equations 
was  applied  to  the  nonlinear  filter  problem  in  [13]  ,  working 
directly  with  the  Zakai  equation  and  its  adjoint  and  allowing 
correlations  between  the  Wiener  processes  W ,W  driving  the  state  and 
observation  equations.  However,  technical  difficulties  are  encountered 
in  adapting  that  method  to  the  control  problem. 

In  [8]  another  "separated"  control  problem  was  considered. 

In  that  formulation  the  "state"  for  the  separated  problem  corre¬ 
sponds  to  the  (normalized)  conditional  distribution  measure  of 
given  past  observations.  Some  of  the  results  in  [8]  are  proved  under 
assumptions  not  satisfied  when  X  is  a  controlled,  partially 
observed  solution  to  (1.1a).  Hence,  the  results  of  [8]  are 
complementary  to  those  in  the  present  paper. 

A  first  existence  theorem  asserting  that  there  is  a  control 
minimizing  J  is  proved  in  §4,  when  F  =  0  in  (1.2).  When  F  +  0 
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different  methods  are  needed.  In  §7  we  assume  that  the  coefficient o 
in  (1.1a)  is  N  *  N  nonsingular,  and  use  methods  of  the  L  -theory 
of  parabolic  partial  differential  equations. 

In  [4]  Christopeit  proved  an  existence  theorem  for  optimal 
stochastic  controls  under  partial  observations.  In  that  work,  the 
observation  process  is  a  deterministic  function  of  (part  of)  the  past 
trajectory  of  the  state  process,  and  the  optimal  control  is  sought  in 
a  class  of  feedback  controls.  Both  our  results  and  our  methods  of 
proof  differ  significantly  from  his. 
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2.  Formulation  of  the  problem.  We  make  the  following  assumptions 
about  the  functions  appearing  in  (1.1). 

(A^)  o  is  a  bounded,  continuous  N  x  D  matrix- valued 

function  on  3R^+^.  Moreover,  o(»,y)  is  Lipschitz  on  IR1^  with 

M 

Lipschitz  constant  not  depending  on  y  €  1R  . 

(A2)  b(x,y,u)  =  b^(x,y)  +  b1(x,y)u,  where  b^b1  are 

n+m  a 

bounded,  continuous  functions  on  1R  .  Moreover,  b  (*,y)  is 
N 

Lipschitz  on  IR  with  Lipschitz  constant  not  depending  on 
y  €  JRM,  for  £  =  0,1. 

Note  that  in  (A^) ,  b^  has  values  in  ]r\  while  b*  has 

N  x  L  matrices  as  values. 

N 

We  write  (LR  )  for  the  space  of  bounded  continuous  real -valued 

functions  on  1RN,  and  CQ(1RN)  for  the  space  of  continuous  func- 

k  N  k  N 

tions  with  compact  support.  We  write  C*R  ),  CQ  OR  )  for  the 
spaces  of  functions  whose  partial  derivatives  of  orders  <  k  are 

in  CbQRN),  C0QRN)  respectively.  Similarly  we  write  Cb0RN;lRM)  , 

if  n  m  M 

CqQR  dR  )  if  the  functions  are  IR  -valued. 

(A3)  h  6  CbORNORM). 

In  §7  we  shall  assume  that  o  is  nonsingular  N  x  N.  One 
could  also  let  b,o,h  depend  on  t,  with  minor  changes  in  the 
results,  and  proofs.  This  would  only  be  a  generalization  in  §7,  since 
in  §'s  2-6  t  can  be  adjoined  as  an  additional  x  component. 

(A^)  ^  is  a  convex,  compact  subset  of  IRL. 

Choose  any  T  >  0  which  will  be  fixed  throughout  the  paper. 

We  formulate  the  control  problem  on  the  "canonical"  sample  space 
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n  =  x  n1  x  n2  *  o3, 

where  are  C([0,T]pRm)  with  m  =  D,N,M  respectively 

and 

=  L2  C [0 ,T] ;  <%). 

The  elements  w  =  (W,X,Y,U)  of  Q  satisfy 

“(t)  =  (Wt(«),XtC«),Yt(w),Ut(w))f  0  <  t  <  T. 

We  give  the  usual  norm  topology;  and  ^3  the  weak 

topology,  which  is  metrizable  and  separable  since  %  is  compact  [2, 
p .  238]  .  Let 


sr  =  o  *  n, 


X  Q. 


whose  respective  elements  are  pairs  (W,X) ,  (Y,U).  Let 
.^.(W)  =  a{Ws,  0  <  s  <  t},  with  _^.(Y)  defined  similarly. 

Let 


^(U) 


o{Vs>  0  <  s  <  t},  Vt 


t 

u  ds. 
h  s 


The  elements  of  these  o-algebras  are  subsets  of  fig,...,fi3  re¬ 
spectively.  However,  we  can  also  regard  thenm  as  o-algebras  of 
subsets  of  ,  or  Q  ,  with  the  obvious  identifications.  For 

example  A  €  ^(X)  can  be  identified  with  fig  *  A  *  x  fij. 


i 


i  tr,‘-  j • 


We  shall  also  use  the  o-algebras 


?t(W) 

X 

**(X) 

II 

rg  p 

^t(Y) 

X 

9t  (U ) 

II 

x 

t 

(0  2 
t 

=  J^(W) 

•  x  ^(U) 


We  note  that  U)  is  the  Borel  o-algebra  of  ft,,  and  thus 

2  2 
is  the  Borel  o-algebra  of  ft  . 


Remark .  Intuitively , by  using  the  indefinite  integral  V  instead 
of  U  in  defining  .^(U),  we  need  not  be  concerned  with  changes 

in  U  on  subsets  of  [0,T]  of  Lebesgue  measure  0.  An  alternative 

to  our  formulation  would  be  to  consider  quadruples  (W,X,Y,V)  instead 
of  (W,X,Y,U),  using  the  uniform  norm  on  V.  By  (A^)  the  control 
enters  linearly  in  b.  Hence,  one  can  write,  in  the  integrated  form 
of  (1.1a), 


fV(Xs>Ys)Usds  -  jV(Xs> 


VdV 


the  right  side  being  a  Riemann-Stielt j es  integral.  This  device 
was  used  in  [9],  but  we  use  here  Ut  instead. 

Distribution  of  (W,X)  conditioned  on  (Y,U).  Let  Y  =  Y. ,  U  =  U. 

be  given  sample  paths  for  the  observation  and  control  processes;  thus 
? 

(Y,U)  €  ft  .  Consider  equation  (1.1a)  with  initial  data  WQ  =  0, 

Xq  =  x.  Assumptions  (A^)  ,  ( A 2 )  imply  the  Ito  conditions.  There  is 
a  solution  to  (1.1a)  which  is  pathwise  unique,  and  hence  also  unique 


rvr-  . 
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in  probability  law.  Let  P  ’  denote  the  distribution  measure  of 
(W,X)  given  (Y,U).  Then  PY’^  lies  in  the  space  of  probability 
measures  on  By  convergence  of  a  sequence  of  probability 

measures  PR  to  P  we  mean  weak  convergence,  namely 

i  g (W , X) dP  +  g(W,X)dP  for  all  g  €  C-ftt1). 

Jo  “Jo1  u 


Lemma  2 . 1 .  ’  depends  continuously  on  x  , Y , U . 


This  lemma  is  essentially  known  (cf.  Stroock -Varadhan  [14]) 
However,  for  completeness  we  outline  a  proof  in  the  Appendix. 

Following  the  motivation  described  in  §1,  the  formal 
definition  of  admissible  control  is  as  follows. 


Definition .  An  admissible  control  77  is  a  probability  measure  on 
(Si^ ,  such  that  Y  is  a  77  ,  {  ^~}  Wiener  process. 


The  projection  (Y,U)  Y  maps  71  onto  Wiener  measure. 

The  definition  of  admissible  control  requires,  in  addition,  that 

|  Usds  be  independent  of  Y^  -  Y^  for  t  <  r  <  T. 

Let  denote  the  set  of  all  admissible  controls  77 .  Given 
a  distribution  u  for  Xq  ,  each  77  €  2{  determines  a  joint 
distribution  measure  P^  of  (W,X,Y,Z)  as  follows.  Define 
PY’U  =  Pj’u  by 


FY’U(A)  =  PY,U(A)dn(x),  A  £ 

Ln  x  1 


We  then  define  P^  on  by 


'4* 
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(2.1)  (dW , dX , dY ,  dU)  =  PY ’U  (dW,  dX)  tt  (dY  ,dU)  . 

O 

The  projection  of  under  (W,X,Y,U)  (Y,U)  is  n .  The 

family  of  probability  measures  P  ’  gives  a  regular  conditional 

distribution  for  (W,X).  If  g(W,X)  is  ^-measurable,  then 

EY,^g(W,X)  is  -measurable ,  where  we  write  EY’^,  E^  for 

— Y  U  0 

expectations  with  respect  to  P  ’  ,P_.  We  then  have  for  any 

O  11 

.^-measurable  'P  with  E^l'l'l  <  00 

(2.2)  E^OM^)  =  EY’U(^),  if  -  a .  s . 


We  define  P^  by 


dP 

(2.3) 

IT 

o 

Z_ , 

l 

with 

Zj  as  in  (1.3). 

Since 

O 

h(x)  is  bounded,  P^fft)  =  E^fZy)  =  1 

For  each  (Y,U) , 

W  is 

— Y  U 

a  P  ’  -standard  Wiener  process,  and 

— Y  U 

X  satisfies  the  stochastic  differential  equation  (1.1a)  P  ’  -a.s. 

O 

With  respect  to  P^ ,  W  and  Y  are  independent  standard  Wiener 
processes  . 

Lemma  2.2.  Let  W^_  =  Y^.  -  j  h(Xg)ds.  Then  IV, W  are  independent 
standard  Wiener  processes  under  P^  and  the  stochastic  differential 
equations  (1.1a),  (1.1b)  hold  P^  -  a.s. 


W  ° 

Proof .  Since  the  pair  (y)  is  a  P^-stcndard  Wiener  process,  of 

dimension  N  +  M,  the  Cameron-Martin-Girsanov  formula  and  (2.3) 

W 

imply  that  (^)  is  a  P^-standard  Wiener  process.  Since  (1.1a) 
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holds  P^-  a . s .  and  <<  Pn ,  (1.1a)  holds  P^  -  a.s;  while  (1.1b) 
holds  by  definition  of  W. 

We  have  defined  as  admissible  control  a  probability 
measure  w  belonging  to  the  class  91  .  Convergence  of  sequences 
71  of  admissible  controls  is  taken  in  the  sense  of  weak  convergence 
of  probability  measures.  91  is  a  metric  space  under  (for  instance) 
the  Prokhorov  metric  [2].  Moreover,  91  is  a  convex  set. 


Lemma  2.3.  91  is  compact  under  weak  sequential  convergence. 


Proof.  Since  every  measure  if  6  91  projects  onto  Wiener  measure 

under  (Y,U)  -*■  Y  and  the  second  component  U  lies  in  the  compact 

(weak  topology)  space  L  ( [0  ,T]  ;  ty)  ,  tightness  of  91  follows  by 

standard  arguments.  Hence  [2,  p.  37]  it  remains  only  to  show  that 

91  is  closed.  Suppose  that  n  -*■  if ,  n  £91.  We  must  show  that  Y 

*  c  n  n 

2 

is  a  n,  {  ^  }  Wiener  process.  Since  projects  onto  Wiener 

measure  for  each  n,  so  does  if.  We  need  only  verify  that  Yr  -  Y 
is  independent  of  &  for  t  <  r.  For  this  it  suffices  that 
for  any  ^-measurable  <J>  £  (n^ )  and  f  £  QR  ) 


4>f(Y  -Y  )dir 
fi  r  L 


f(Yr-Yt)dif. 


But  this  holds  for  each  if  ,  and  we  pass  to  the  limit.  This  proves 

n 

Lemma  2.3. 

In  §6  we  shall  consider  the  subclass  91s  of  strict-sense 


controls . 


3.  The  unnormalized  conditional  distribution.  We  wish  to  introduce 
an  unnormalized  conditional  distribution  of  given  controls 

and  observations  up  to  t.  Let  us  take  a  version  of  the 

O 

P^-martingale  Z  such  that  Z  is  ig^-measurable  for  0  <  t  <  T. 
Consider  any  f  €  C^QR^).  By  (2.2)  with  V  =  ffX^jZ^, 

,2  -Y’U 

(3.1)  E7I(f(Xt)Zt|  =  E  (f(Xt)Zt),  0  <  t  <  T,  it  -  a.s. 


Let  us  rewrite  (3.1)  in  such  a  way  that  it  is  defined  for  all 

Y,U,  not  just  77  -  a.s,  and  depends  continuously  on  (Y,U).  See 

2  N  M 

Lemma  3.2  below.  Since  h  €  QR  pR  ) ,  we  can  integrate 


h (X  ) • dY  by  parts : 
-v  s  s 


f'h(Xs)-dYs  =  h(Xt)-Yt  •  |%s.|.sh(Xs)ds  -  (‘vs-YhCXs)a(Xs,Ys)dWs, 


where  Ys*Vh  is  the  gradient  in  x  of  Ys‘h  and 

<3-2>  Ls  ■  I  .  !  +  T.MX'WiIt  • 

1,J*1  J  1  J  1  =  1  1 


with  a  =  oo' .  For  fixed  Y,U,  yg  +  L^  is  the  backward  operator 


corresponding  to  (1.1a).  Let 


(3.3)  e(s,x)  =  \  (aYs-Vh,Ys-Vh)  -  Ys • Lgh  -  \  |h|2, 


2  N 

where  in  (3.3)  (a£,5)  =  \Zo\  denotes  the  dot  product  in  ]RN  of 

M 

a£  with  and  •  denotes  the  dot  product  in  ]R  .  From  (1.3) 


,  ■ 
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v 

exp  Yt-h(Xt)  exp 


ft 

e(s,X  )ds 

0 


where 


v 


Z 


t 


Ys-Vh(Xs)o(Xs,Ys)dW 


•t 

(a  (X  , Y  ) Y  •  Vh  (X  ) , Y  •  V  h  (X  ) )  ds . 

q  b  b  b  b 


For  fixed  (Y,U)  let  us  define  another  probability  measure 
on  (fi1 ,  <fj)  by 


(3.4) 


dVrY’U 


dP 


Y,U 


This  corresponds  to  a  change  in  drift  coefficient  in  equation  (1.1a) 
v 

from  b  to  b  =  b  -  aY  "Vh,  and  changes  L  in  (3.2)  to  the 

s  s 

ope  rator 


(3.5) 


Ls  ■  Ls  -  (aYs-7h,V). 


From  (3.1)  we  then  have 


b7T(f(Xt)Zt|  =  F’U(f(Xt)exp(Yt-h(Xt))  exp  (s  ,Xs)ds)  , 


where  the  right  side  is  now  defined  for  all  (Y,U)  €  n  ,  not  merely 
ff-a.s.  For  fixed  (Y,U)  the  right  side  is  a  bounded  linear  func¬ 
tional  on  @R^) .  Hence,  for  every  (Y,U)  6  and  0  <  t  <  T 
there  exists  a  measure  AY,U  on  the  Borel  o-algebra  ^QRN)  such  that 

(3.6)  <f,A^U>  =  ^Y,U(f(Xt)exp(Yt-h(Xt))  exp  j\;(s  ,Xs)ds)  , 
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for  all 


f  £  (1R  )  where  for  any  measure 


v  with 


_,N. 
v(K  ) 


<  00 


<f  ,v> 


r 


Nf (x)dv(x) . 
!1Rn 


Y  U 

Defini tion .  A  ’  is  the  unnormalized  conditional  distribution 
measure . 


The  unnormalized  conditional  distribution  measure  satisfies, 
for  all  f  €  Cb0RN) , 

(3.7)  <f,AY’U>  =  E1T(f(Xt)Zt| 


As  is  well-known,  the  (normalized)  conditional  distribution  of 
satisfies,  for  all  f  €  CbQRN), 


E1T(f(Xt)|  $2t) 


<f,AY’U> 


t 


where  E^  denotes  expectation  with  respect  to  the  measure 
defined  by  (2.3).  For  fixed  t,  let 


(3.8) 


Y,U 


pY ,  U 


v q  u ( x )  =  E;’u(f(Xt)exp(Yt-h(Xt))exp 


e(s,X  )ds), 

J  o  s 


VY  U 

where  E  ’  denotes  expectation  with  respect  to  the  probability 
measure  PY,U  in  (3*4)  for  state  XQ  =  x.  For  initial 

distribution  u  for  Xg  , 


EY'U 


(gtx))  ■  f  ^’u(g(X))du(x). 
J  w  x 
TR 
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Therefore,  by  (3.6),  (3.8) 

(3.9)  <f,AY’U>  =  <Vq  ,l1  ,y  > 

for  all  (Y,U)  €  ft2  and  f  €  Cb(IRN). 

In  55  we  shall  see  that  (3.9)  has  a  natural  interpretation 
in  terms  of  solutions  to  forward  and  backward  partial  differential 
equations . 

Y  U 

Remark .  We  shall  later  wish  to  consider  A  *  corresponding  to 

any  p  >  0  with  p  0R^)  <  ”,  not  merely  for  probability  measures  p 

on  .#QR^) .  Given  Y,U,  and  P,  the  right  side  of  (3.9)  is  a 

bounded,  nonnegative  linear  functional  of  f,  by  (3.8).  This 

Y  U 

gives  an  alternate  way  to  define  the  measure  '  without  the 
restriction  pQRN)  =  1,  in  such  a  way  that  (3.9)  holds. 


Y  U 

Lemma  3.1.  (a)  Vq ’  (x)  is  a  continuous  function  of  (x,Y,U) 

(b)  Given  f  e  CgQRN)  and  a  >  0,  there  exist 


c ,  k  > 

0  (depending  on  f.a  and  bounds  for 

1  b  |  ,  |°|, 

| Vh | )  , such 

that 

lvY,U(x)l  <  c  exp(-kx)  for  all 

x  e  irn 

and  Y,U 

such  that 

INI 

<  a . 

Proof . 

By  Lemma  A.l  (Appendix)  )^Y’U 

depends  continuously  on 

x  ,  Y ,  U . 

Let  xn  "  x’  Yn  Y’  Un  "  U’ 

and  let 

(for  fixed 

t) 

VX)  =  f(xt^exP(Ynt'h(Xt))exp 

ft 


Y(X)  =  f(Xt)exp(Yt*h(Xt))exp 


0 


j‘en(s.Xs)ds 

e(s,Xs)ds, 
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where  e  is  defined  by  (3.3)  with  Y,U  replaced  by  Y  ,U  .  For 
n  *  *  n  ’  n 

N 

any  compact  T  c  C([0,T);3R  ),  e  ds  -*■  eds  as  n  -*■  «,  uniformly 

Jo  J0 

on  r.  This  is  proved  by  the  same  reasoning  used  in  the  proof  of 

Lemma  A.l.  Then  Y  -*■  Y  uniformly  on  T .  From  this  we  conclude 

n  ' 

Y  ,11  V  u 

that  v0  (x  )  vq  ’  (*)  as  n  -*•  00 ,  which  proves  (a). 


To  prove  (b)  ,  l!  is  bounded  by  (A^).  For  ||Y||  <  a, 

Yt’h(X  )  and  e(s,Xg)  are  bounded.  Hence,  for  some  Cj , 


Y,U 


(x)i  -  e  spt  n 


vy  U 

However,  P  ’  -  a.s. 

'  x 


dX  =  b(t,Xt)dt  ♦  odWt, 


v  [* 

=  b  -  aYs-Vh,  Wt  =  Wt  ♦  j  Ys-vh(Xs)o(Xs,Ys)ds 


v  VY  U 

and  W  is  a  P  ’  -Wiener  process.  For 


<  a,  b  is  bounded; 


and 


o  is  hounded  by  (A^).  By  standard  estimates 


IVXI  <  c2  IKK  ♦  c-t, 


odW  , 


0 


F'U(Xt  €  spt  f)  <  P^U(|  M|  >  kj  |  x  |  -  k2) 


for  some  k^  >  0  and  k 2  (){  j|  is  as  usual  the  sup  norm).  Using 
the  fact  that  o  is  bounded  and  an  exponential  martingale  inequality, 


T 


for  some  c^,  k  >  0.  See  for  example  [14,  p.  87).  This  proves  (b) . 
For  r  >  0  let 

=  {p  >  0  on  :  ||p||  <  r}  , 

where  ||p|||=  pQR^).  We  use  vague  convergence  for  sequences  of 

N 

measures:  v  -*■  v  means  that  <f,v  >  -*■  <f,v>  for  all  f  €  C„  QR  ). 

n  n  '  0  ' 

Y  U 

The  next  lemma  asserts  continuous  dependence  of  A  '  on  P,Y,U 
for  fixed  t,  provided  we  restrict  p  to  (see  the  Remark 

preceding  Lemma  3.1). 

YU  2 

Lemma  3,2.  A  ’  is  a  continuous  function  of  p,Y,U,  on  jtr  *  n  . 

Proof.  Let  p  p  ,  (Yn,U  )  ->  (Y,U).  Given  f  e  CQ0RN)  let 

Y  ,U  ..  .. 

vn(x)  =  v0n  n(x),  v(x)  =  v0'  (x). 

By  (3.9)  it  suffices  to  show  that 

(*)  lim  <vn»yn>  =  <v,P>. 

n-*-°° 

By  Lemma  3.1(a),  x  -*■  x  implies  v  (x  )  ■*  v(x).  Hence  v  -*>  v 

n  r  n  n  n 

uniformly  on  compact  subsets  of  1RN.  Since  Y^  -+■  Y,  j  | |  |  <  a 

for  some  a.  By  Lemma  3.1(b),  vn(x)  0  as  )x|  -*■»,  uniformly 
with  respect  to  n.  Since  ||pn||  5  r>  this  implies  (*)  and  hence 


Lemma  3.2. 
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4.  The  "separated"  control  problem.  As  in  (1.2)  let 


(4.1) 


J(") 


fT 

{  F(x.,u.)dt  +  C.(X  )}, 
0 


with  the  expectation  with  respect  to  the  probability 

measure  in  (2.3).  The  minimum  problem  is:  given  a  distribution 

measure  M  for  Xg ,  find  a  control  n*  £  81  such  that  J  (n*)  <  .If71) 
for  all  77  €  ?l  . 

(A^ )  F,G  are  continuous,  and  F  >  0,  G  >  0.  There  exists 
77  €  91  such  that  J(n)  <  00 . 


We  sometimes  impose  the  stronger  condition: 

(A^)  F,G  are  continuous,  F  >  0,  G  >  0,  and  for  some 
positive  C,m,  l  >  m 


|  F  ( x  ,  u )  |  <  C  ( 1  +  |  x  |  m )  ,  |  G  ( x )  |  <  C  ( 1  +  |  x  |  m )  , 


<  I  X I  , u >  <  ». 


Since  X^.  satisfies  the  stochastic  differential  equation 
(1.1a)  with  bounded  coefficients  b,o,  J(77)  <  00  for  all  77  €  ?I 
provided  that  (A£)  holds.  See  [10,  p.  48]. 


From  (2.3)  and  the  fact  that  xt»ut  are  -measurable 

T 

Jt1*)  =  F  t,  ( |  ZtF(Xt,Ut)dt  +  ZtG(Xt)  }  . 


Upon  taking  conditional  expectations  and  using  (3.7) 


(4.2) 


JO7)  =  En{foE7I(ZtF(Xt,Ut)|  ^)dt  ♦  I„(ZtG(Xt)|'^ 
T 

<F(-,Ut),^’U>dt  ♦  <G,A**U>)d"(Y,U). 


)>. 


m  = 
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In  the  separated  problem,  we  regard  the  unnormalized 

Y  U 

conditional  distribution  measure  A  =  A  '  as  the  "state", 
and  (4.2)  as  the  criterion  to  be  minimized.  Initially,  Aq  =  u. 

The  dynamics  of  the  measure -valued  process  A  will  be  described 
in  §  5 . 

In  our  formulation,  the  separated  control  problem  is 
completely  equivalent  to  the  problem  originally  formulated  in  §2. 

An  optimal  control  n*  for  either  problem  is  also  optimal  for 
the  other. 

In  the  case  F  =  0  we  can  now  prove  the  existence  of  an 
optimal  n*.  In  §7  we  shall  prove  another  existence  theorem,  with 
F  t  0,  using  methods  of  partial  differential  equations.  One 
cannot,  in  general,  reduce  F  f  0  to  F  =  0  by  adding  a  new 
state  variable  since  linearity  would  then  no  longer  hold  in  (A2) ,  §2. 

Theorem  4.1.  Let  F  =  0.  There  exists  n*  £  ?1  such  that 
«.  for  all  "  £  ?J . 


Proo f .  By  Lemma  2.3,  ?l  is  compact.  It  suffices  to  show  that 


•>(")  = 


<G,A^’U>d"(Y,lJ) 


n 


N  1 

is  lower  sem  icont  inuous  on  $1 .  For  P  £  C  ^  (TR  ),  H  £  OR  ), 
0  <  p  <  1,  I!  >  0 ,  let 


J(") 


0 


II  [ <PG 

2 


,A^’U>]d"(Y,U). 


5jr  -• 


By  Lemma  3.2,  with  u  fixed,  the  integrand  is  a  bounded  continuous 
function  on  S2  .  Hence  J  is  continuous  on  $1 .  Let  P  =  Pn , 

H  =  H  be  increasing  sequences  such  that  Pn(x)  1>  Hn(z)  2  aS 
z  -*■  Then  JC71)  is  the  limit  of  the  corresponding  increasing 
sequence  JnCIt),  which  implies  that  JC71)  is  lower  semicontinuous 
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5.  Dynamics  of  A  .  We  begin  by  imposing  rather  stringent 
regularity  conditions  on  the  coefficients  in  (1.1),  and  by  assuming 
that  the  initial  distribution  u  has  a  density  pQ  £  c”  0RN) .  Then  A^ 
turns  out  to  have  a  density  q(t,x)  which  obeys  the  Zakai 
stochastic  partial  di f ferential  equation  ,  as  in  case  of  nonlinear 
filtering.  However,  it  is  more  convenient  to  consider  instead 
p  =  q  expC-Y^-h),  which  obeys  the  partial  differential  equation 
(5.4).  Later  in  the  section  we  drop  the  regularity  assumptions, 
and  obtain  the  same  equation  in  a  weak  form. 

2 

The  regular  case.  We  fix  (Y,U)  £  fi  ,  and  for  the  present  assume 
that  U  is  continuous  on  [0,T].  We  also  assume  for  the  present 
that,  for  fixed  Y,o,b°,b*,h  are  of  class  C^QRN).  Given  t  >  0 
and  f  £  C^flR''1)  consider  the  following  "backward”  partial  differ¬ 
ential  equation 


(5.1)  ^  +  Lsv  +  e(s)v  =0,  0  <  s  <  t, 

v ( t )  =  f  exp(Yt -h)  , 

v 

where  we  have  written  v(s),e(s)  for  v(s , • ) ,e (s , • )  and  Lg  is 
defined  by  (3.5).  The  Cauchy  problem  (5.1)  has  the  (unique) 
probabilistic  solution 


(5.2)  v(s,x)  =  E^U[f (Xt)exp(Yt -h(Xt))exp  j  e(«,XQ)d6], 

vY  U  .  v 

P*  is  the  distribution  measure  of  (W  ,X  )  satisfying 

A  L  L 


where 
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dX  =  Sdt  + 
pY  >U  =  vY,U 
ox  x 


adXrt ,  s  <  t 
)•  By  (3.8) 


<  T, 


with 


X 


s 


x  (in  particular, 


(5.3)  v(0)  =  v*’U. 

oo  N 

Under  our  regularity  conditions,  v(s)  £  C  0R  )  for  0  <  s  <  t. 

This  follows  from  the  smooth  dependence  on  the  initial  state  x  of  path 

wise  solutions  X  to  dX.  =  f>dt  +  odW .  ,  X  =  x,  with  W,.  a  fixed 

t  t  s  t 

Wiener  process  on  some  (fi,(  ^  } ,P) .  By  essentially  the  same  proof 
as  [10,  p.  74]  dv/ds  is  continuous  and  (5.1)  holds.  Moreover, 
each  partial  derivative  of  any  order  of  v  in  the  variables 


X,  ,  .  .  .  ,x 

1  ’  n 

tends 

to 

0  exponentially  as  |x|  -*-00.  For  instance  by 

replacing 

X  by 

X 

VY  II  — 

and  Esx  by  E  =  E-  in  (5.2),  and  differentiat 

ing  with 

respect 

to 

x^ ,  we  get  an  estimate 

|v 

1  X 

(s,x)f  <  C  max  E(xc|C.  (t)  |] 
i  s<t<t  t  1 

with  =  3X/9x^  and  the  indicator  function  of  the  event  X 

Xt  £  spt  f.  By  [10,  p.  61],  E|t^(r)p  is  bounded  (independent  of 
1  and  x)  for  each  p  >  0.  By  taking  p  =  2  and  using  Cauchy- 
Schwartz  we  get 


|vx  (s,x) |  <  C1[P(Xt  £  spt  f)]1/2. 


Since  P(X  €  B)  =  $I’^(X  £  B) ,  the  proof  of  Lemma  3.1b  then  shows 

t  o  A  L 

that  v  (s,x)  -*•  0  exponentially  as  |x|  -*00.  Similarly,  higher 
xi 

order  derivatives  of  v  tend  to  0  exponentially  as  |x|  ■*■<*>,  using  the 
fact  that  partial  derivatives  of  X  of  all  orders  with  respect  to 
xl'...*Xn  ^ave  hounded  expectations  [10,  p.  61]. 

Let  us  also  consider  the  following  initial  value  problem  for 
the  equation  adjoint  to  (5.1): 
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(5.4)  =  L*p  +  e  (t)p ,  t  >  0, 

P(0)  =  PQ, 

where  p^  e  Cg0R^).  The  time  reversal  s  =  T  -  t  changes  (5.4) 

v 

into  a  problem  of  the  same  form  as  (5.1),  but  with  Ls  replaced  by 
another  degenerate  parabolic  operator  l'  and  e(s)  by  another 
e' (s)  .  Therefore,  (5.4)  has  a  unique  solution  with  p(t)  e  C°°(IR^) 
and  with  all  partial  derivatives  of  any  order  in  x^,...,x^  tending 
to  0  exponential ly  as  |xj  +«>. 

Let  us  write  (  ,  )  for  scalar  product  in  L^QRN). 
Integrations  by  parts  imply  (v(t),p(t))  =  constant.  In  particular, 

( v ( t ) , p ( t ) )  =  (vJ’U,p0). 

If  P0  is  the  density  of  m,  then  we  have  from  (3.9)  since 
v(t)  =  f  exp(Y-h) 

(5-5)  I  P (t ,x) exp (Y*h(x))f(x)dx  =  <f , »U> . 

J  M  Z  t 

1R'n 

Let 

(5*6)  q(t,x)  =  p (t , x)exp  (Yt • h  (x) ) . 

Y  U 

(of  course,  q  =  q  ’  depends  on  the  observation  and  control 
trajectories.)  Then  (5.5)  implies  that  q(t)  is  the  density  of 
the  unnormalized  conditional  distribution  measure  =  A^’^, 
under  the  above  regularity  assumptions.  The  partial  differential 


equation  (5.4)  determines  the  dynamics  of  p(t),  hence  also  of 

q(t) . 

Equation  (5.4)  is  a  linear  partial  differential  equation 
in  which  the  processes  Y,U  enter  parametrically.  In  contrast, 
the  Zakai  equation  for  q(t),  see  (5.8)  below,  is  a  stochastic 
partial  differential  equation  driven  by  the  Y  process.  The 
technique  of  replacing  the  Zakai  equation  by  (5.4)  is  analogous 
to  the  technique  of  Doss  [7]  and  Sussmann  [15]  for  reducing 
certain  finite  dimensional  Ito-sense  stochastic  differential 
equations  to  ordinary  differential  equations  depending  paramet ri 1 1 
on  a  Wiener  process.  The  same  idea  has  been  used  in  nonlinear 
filtering  by  Liptser-Shiryaev  [12],  Clark  [5],  and  others.  See 
Davis  [6 ] . 

The  general  case.  Let  us  return  to  the  assumptions  (Aj)-(A3)  on 
o , b ^ , b ^ , h .  We  consider  fixed  (Y,U)£ft2,  and  any  distribution  y 
for  Xq.  Let  us  rewrite  (5.4)  in  a  weak  form.  Define  the  measure 

A  t  by 

(5.7)  <g,At>  =  <g  exp(-Yt-h)  ,At> ,  g  £  Cb(IRN). 

In  the  regular  case,  A  has  density  p(t).  By  multiplying  (5.4) 
by  g  £  Cq(IRN)  and  integrating  by  parts,  we  get 

,4  ~  V  ~  -v  co  M 

(5.4')  ^  <8>At>  =  <Ltg,At>  +  <e  C  t  ^  g  ,  At=> ,  g  £  CQ  OR  ). 

This  is  the  weak  form  of  (5.4).  The  initial  data  are  now  AQ  =  y. 


f  ♦ 
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2 

Theorem  5,1.  Equation  (5.4')  holds,  for  any  (Y,U)  £  Q  ,  any 

'J  M 

g  £  0R‘  )  and  initial  distribution  M  for  Xq . 


Prop f .  For  g  £  C^QR^)  (5.4')  holds  in  the  regular  case.  For  fixed 
0  1  00  N 

Y,  take  o  ,b  ,b  ,h  of  class  C,  QR  )  ,  uniformly  bounded  and 
’  n  n  n  n  b 

tending  uniformly  to  a,b^,b*,h  as  n  °°  with  partial 
derivatives  o  ,bp  ,bY  ,h  uniformly  bounded.  Moreover,  let  U 

1  1  1  1  J  " 

tend  to  U  almost  everywhere  on  [0,T],  U  £  <2r,  and  tend 

weakly  to  u ,  where  U  is  continuous  on  [0,T]  and  u  has 

n  y  T  i  n 

oo  xj  y  v 1  *  un 

density  p  £  CnQR  ).  Let  P  =  Pnv  ,  where  the  subscript  n 

il  O  U  II  a  11a 

z  VI 

means  that  c,b  ,h  are  replaced  by  °  ,bn,h  ,  x.  =  0,1. 

v  v  y  n 

Lemma  A.l  implies  that  P  -»  P  ’  if  x  -*•  x  as  n  -+  «>.  Let 

n  x  ii 

N  11 

f  ^  Cq QR  ).  The  same  proof  as  for  Lemma  3.1(a)  implies  that 


v  (x) 
on  v  J 


Exn^f  (-Xt-)exp(-Yt*h(Xt^exp 


e  (s ,X  )ds) 

0 


tends  uniformly  on  any  compact  set  to  Vq’^(x).  Moreover,  by 

Lemma  3.1(b),  vQn(x)  -»  0  as  |  x  |  -»  00  uniformly  with  respect  to  n. 

Let  A  be  the  corresponding  unnormalized  conditional  distribution, 

with  A  =  )i  .  Ry  (3.9) 
no  n  ’ 


<  f ,  A  > 
’  nt 


=  <v0n 


<f,At> 


=  <vQ,y> 


where  vQ  =  Vq,U.  Then  <von’yn>  <vq»P>-  Since  this  is  true 
for  every  f  €  Cq(TRN)  ,  A  t  ->  At  vaguely  as  n  -*•  »  with  |  |Antj  | 
bounded.  From  (5.7),  A  t  -*■  At  vaguely  with  II  Ant  I  I  bounded. 

We  rewrite  (5.4')  in  the  regular  case  in  integrated  form: 


t 
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M  V 

For  each  g  £  c“(IRN),  Lsng,  en(s)  are  uniformly  bounded  and  tend 
to  Lgg,e(s)  uniformly  on  1R^,  for  almost  all  s  £  [ 0 , T ]  - 
By  passing  to  the  limit  we  get  (5.4'),  when  g  £  c”(1RN),  Finally, 

we  approximate  g  £  C^(IRN)  by  gn  £  Cq0RN)  such  that  gn»bsg 

v 

are  uniformly  bounded  and  tend  to  g,Lsg  as  n  -*■  °° ,  uniformly  on 
compact  subsets  of  3RN.  By  passing  to  the  limit  in  (5.4')  we  get 
Theorem  5.1. 

We  do  not  have  a  uniqueness  result  for  equation  (5.4'),  in 
contrast  with  the  nondegenerate  case  to  be  considered  in  57.  More¬ 
over,  in  §  7  we  will  be  able  to  use  results  from  the  theory  of 
parabolic  PDF  concerning  the  continuous  dependence  of  solutions 
on  the  coefficients  to  get  a  stronger  existence  theorem  for  an 
optimal  stochastic  control. 

The  Zakai  equation.  The  unnormalized  conditional  distribution 
satisfies  the  following  (Zakai)  equation,  written  in  a  weak  form. 
Recall  that  Y  is  a  P^  ,  {  -brownian  motion  for  every 

admissible  control  n . 


Theorem  5.2.  For  every  f  £  Cj^  QR^) 


(5.8) 


d<f,At>  =  <Ltf,At>dt  +  <hf,At>-dY  . 


Prop  f .  Let  ^(t,x)  =  f (x)exp(Yt *h(x) ) .  Then 


<f  ,At>  =  « P  (t)  ,At>  , 


>  '  *rT 
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where  as  before  we  set  ^(t)  =  ^  ( t ,  • )  •  For  fixed  x,  the  Ito 
differential  rule  implies  that 


<W  =  \  Y|h| 2dt  +  ^h-dY. 


Fiven  t  >  0 ,  we  partition  [0,t]  into  m  subintervals  [t^  j,t.] 
of  length  m  ^t.  Then 


:,A  >  -  < f , A  >  =  l  «Ht.),A  *  L  > 

t  j  =1  J  J-l 

m 

+  l  <tp  C  t  -  )  -  ip(t  1  )  ,  A  > 
j=l  J  3  1  tj-l 


m  t  j  v 

E  [<LJ'(t.)  +  e(s)*(t.),A  >]ds 

j=lJt.  ,  S  3  J  5 

J  -1 


5?  rti 


HI  J  n  7  ~ 

+  E  J  (s) [ h I  ,  A  >ds 

i'1 

t  . 
m  ,  ] 

+  l  (s)h,A  > ' dY  . 

j“lJ  t^  _!  J-l 

(To  justify  the  exchange  of  stochastic  and  Lebesgue  integrals  see  the 
Note  below.)  For  fixed  Y,U, 


<4)  (s)h ,  A  >  -  <^(s)h,A  >| 

J-l 


b-i 


[<L9(*(s)h),A6>  ♦  <e(6)iKs)h,A9>de| 


<  c(s-t.  .)  <  cm"iT, 
j-l  -  ’ 
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where  c  depends  on  Y,U.  Hence  as  m  -*■  »  the  last  term  tends  in 

O  *  t  ^ 

P^-probability  to  <'(J(s)h,A  >-dY  .  By  using  a  similar  estimate 

0  s  s 

for  the  integrand  in  the  middle  term,  and  elementary  estimates  for 
the  first  term,  we  get 


<f,At>  -  <f,A0> 


=  <L>Ms)  +  e(s)iKs)  +  \  <Ks)  |  h  |  2,A  >ds 
n  L  s 


+  <iKs)h,A  >-dY  . 
j  0  s  s 


A  straight  forivard  calculation,  using  (3.2),  (3.3),  (3.5)  gives 


ex? (Yg ' h) LR  f  =  Ls^(s)  +  e(s)*(s)  +  j  ^  (s ) | h | 2 . 


Moreover,  from  (5.7) 


<exp(Ys*h)Lsf  ,Ag>  =  <L(.f,A<.> 


s  '  s 


<1'(s)h,A  >  =  <exp(Y  -h)hf,A  >  =  <hf,A  >, 


Therefore 


<f,At>  -  < f , Afl >  =  J^<Lsf,As>ds  +  |  <hf,As>-dYc 


This  is  the  integrated  form  of  (5.8),  and  proves  Theorem  5.2. 


Note .  In  the  proof  we  have  used 


(5.9) 


t . 

J 

(s ) h , A>  *  dY  =  <;,A> 

Vi 


where  for  brevity  we  now  write 
on  1RN) 


A  and  where  (pointwise 


J  (s )  h  •  dY  =  4>(t.)  -  l|>(t.  ,)  -  \  f  ^  Hs)  |h|2dx. 
t,  ,  3  3  “  Jt,  , 


The  functions  £,^(s)h  are  bounded  and  uniformly  continuous  on  ]R 

The  bounds  and  moduli  of  continuity  depend  on  f  and  |  | Y [ |  ,  but 

not  on  s.  For  n  =  1,2,...,  partition  Bn  =  {|x|  <  n)  into 

Borel  sets  A?,...,An  of  diameter  <  n  *  and  choose  x11  £  A?. 

1  mn  11 

Then 


t . 

(5.10)  [  J  [l  ns,xJ)h(xJ)A(A?)].dY_  =  l  «(x?)A(Aj). 

Jt.1i  i 

J  -1 


For  each  (Y,U),  the  right  side  tends  to  <Q,A>  as  n  -*■  ®.  The 
sum  in  brackets  on  the  left  side  tends  to  <^(s)h,A>  uniformly 
with  respect  to  s.  Hence  the  stochastic  integral  converges  in 
probability  to  the  left  side  of  (5.9)  as  n  [10,  p.  11,  IV]. 

This  proves  (5.9)  . 


Theorem  5.3.  For  K  =  1,2,... ,m  >  0 

m  m 

Elt(<(l+|x|2)I,At>K)  <  C<(l+|x|2)7,P>K, 


(5.11) 
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where  C  depends  on  K , m ,  and  t  (but  not  on  11  €  21 )  . 

Proof.  For  0  <  a  <  1 ,  let 

m 

fa(x)  =  (1  +  |  x|  2)7exp[-a  (1+ |  x|  2) 1//2]  . 

Any  easy  calculation  shows  that  fa  6  C^QRN)  and  |Lsfu|  5  Cjfa 
for  suitable  depending  on  m.  The  Zakai  equation  (5.8)  and 

Ito  differential  rule  imply 


d<fa,At>K  =  [K<fa,At>K'1<Ltfa,At> 

*  K(K-l)<fa,At>K'2|<hfa,At>|2]dt 
+  K<hfa,At>K'1.dYt. 

For  a  >  0  let  t  =  inf(t:  | | A  | |  >  a}.  From  (3.6)  with  f  =  1, 

cl  L 

||At||  =  <1,A^>  is  continuous  in  t  and  } -adapted .  Hence 

is  a  stopping  time.  Let  x  be  the  indicator  funct ion  of  the  set 

cl 

(s  <  x^}.  Then 


H7T<fa’AtAi  > 
a 


K 


=  <  f  .u  > 
a 


*  K’E*  f'vfa.VK"1<Lsfa.V‘)s 

*  K(K-1]E„  rxa<fa,As>l£-2|<hfa.As>|2ds. 


We  have  since  fa  >  0  and  |Lsfa|  <  C^fa, 


A 
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<Lsfa>Vl  ^  C1<fa,As>i 


l<hfa,As>|  5  INI  <  fa,A  >, 


ft 


ETT<fa«AtAT  >K  -  <fa^>K  +  (KC1  +  K(K-l)j  |h|  l2)^  j \g<fa ,  As>Kds 

3  ^  0 


However,  Xa<f(x,As>  <  cf^A  T  >.  Gronwall's  inequality  then  implies 


E7T<Eot > AtAt  >  5 


C  =  exp[(KC1  +  +  K(K-l)  |  |h|  |2)t] 


We  let  a  -*  °°  and  then  a  -*■  0  to  obtain  (5.11). 


- 


.-2 


6.  Strict -sense  admissible  controls.  We  recall  the  notations 
of  §2. 

Definition.  We  say  that  w  €  SI  is  a  strict  -sense  admissible  control 
if  there  exists  u:  ^  ^3  such  that  u  is  (  ^(Y)  ,  ^  ( U ) ) 

measurable  for  0  <  t  <  T,  and  for  every  ^-measurable  v  >  0 

HY,U)d*  =  f  ^  (Y  ,u(Y)  )dw, 

2  J 

a  n2 

where  w  is  Wiener  measure  on  -%.(Y)). 

For  any  n  £  31 

"(dY.dU)  =  irY(dU)w(dY)  , 
y 

where  tt  is  a  regular  conditional  distribution  for  n .  Strict- 
sense  admissible  controls  are  those  such  that  =  6^^,  w  -  a.s., 

where  6^  =  Dirac  measure  on  (Jlj ,  j£j.(U)  )  concentrated  at  u.  By 
admitting  in  §2  controls  n  €  2i  which  are  not  s  tri  ct  -  sense ,  we 
are  in  effect  allowing  the  choice  of  to  depend  on  auxiliary 

randomizations.  Let 

?F  =  {strict-sense  admissible  n}. 

Corresponding  to  ^  £  ?IS  there  is  a  causal  functional  Y  such 
that  U  =  Y(t,Y)  Lebesgue  x  ^-almost  everywhere  f 1 6 ] .  Causal  is 
in  the  sense  that  Y  =  Y^  for  0  <  s  <  t  implies  Y(t,Y)  =  Y(t,Y') 
for  Y,Y'  €  C([0,T)dRM).  (We  do  not  use  this  result  in  this  paper.) 

It  can  be  shown  that  3iS  is  dense  in  31.  We  shall  not  prove  this 
here.  However,  we  shall  show  that  the  infimum  of  J(ir)  on 


3  2 

is  the  same  as  on  Si  (Theorem  b.l).  I:or  this  purpose  we  consider 
approximations  by  piecewise  constant  controls. 

For  m  =  1,2,...  let  us  partition  [0,1]  into  m  equal 
sub  intervals  [t.  j  ,  t  .  )  ,  t^.  =  j  A,  A  =  in  *T.  Let 

ft.  =  {U  £  ft.:  I).  =  constant  on  ft.  ,,t.),  i  =  1  .....  in }  . 

JIM  .it  J  *  1  1  ’ 


On  weak  and  strong  convergence  of  a  sequence  arc  both  equivalent 


to  pointwisc  convergence  on  each  subinterval 


t .  j  ,  t  .  )  .  He  f i nc 


:  <2. 

m  3 


'  3m 


by  1  (U)  =  IJ  where 
m  m  ’ 


0,  0  <  t  <  A 


2 

As  m  -*■  °° ,  0  in  L  -norm,  for  every  U  6  ft^.  l.et 

0“  =  0.  x  0,  ,  and 
m  2  3m 


?l  =  C  SI  :  Ho2)  =  1  }  . 
m  m 


If  tt  G  m ,  t  £  [t.,tj+j),  then  Wj.  is  independent  of  the 

increments  Y  -  Y  for  t .  <  s  <  r  under  n  . 
r  s  j  - 

We  call  ^  (Y,U)  strongly  continuous  on  ft  *■  =  *  ftj  if 

4>  is  continuous  when  ^  has  the  L2-norm  topology  rather  than 
the  weak  topology.  We  also  denote  by  tf  the  mapping  from 

2  i 

n  ■*  such  that  (Y,U)  -►  (Y,4>  (U)). 


r 


✓  . 
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Lemma  6.1.  Let  ^  be  bounded  anJ  strongly  continuous  on  .  Let 

7T„  =  7i.  Then 

m  m  - 


lim  f  VCV,U)d7i  =  f  *(Y,U)dn. 
m-»«  m  J  2 

m  U 


Proof .  By  definition 


*(Y,U)dTt 
?  m 


,4>  U)d7I. 
m  1 


Since  <I>m(U)  -+  U  strongly,  the  lemma  follows  from  the  dominated 
convergence  theorem. 

In  particular,  we  may  take  in  Lemma  6.1  any  Y  bounded 
and  continuous  on  Si",  where  has  the  weak  topology.  Thus: 


Coro  1  lary  6.1.  As  m  -*  °° ,  4>  n  n  >  for  ever v  n  g  , 

Let  ?ls  =  ?1  n  ?is. 

m  m 


Lemma  6 . 2 .  Let  v  be  bounded  on  ft  and  continuous  on  any 
compac t  subset  of  ft~.  Then 


i  n  f 


ydn  = 


inf 

,s 


d  n 


?!' 


m 


We  leave  the  proof  of  this  lemma,  which  depends  on  standard 
but  tedious  arguments,  to  the  Appendix. 

In  addition  to  (Aj)-(A4)  in  §2  we  assume  lA^t  m  I  . 

We  use  the  "separated"  formula  (4.2)  for  J  ( n )  . 


<  . 
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Theorem  6.1.  inf  J(TI ) 


inf  J(ti)  . 
91s 


Proo f .  Since  91s  c  ft,  we  have  <  .  Let  P  £  CqQR^)  with 
0  <  P  <  1  ,  I!  £  C^OR1  )  ,  and 


V  0  ,11) 


1 11  [  <  P 1  c  •  ,11  )  ,h\  ,ll>  )dt  +  H[<CG,A>..’U>|  , 
0 


J( 71 ) 


tR  d  TT  . 

? 


By  Lemma  .3.2,  r  satisfies  the  hypotheses  of  both  Lemmas  6.1 

and  n.2.  Hence,  for  every  e  >  0,  and  r  £  ?l  there  exist  m  and 

;I ,  £  91'  such  that 

1  m 


•H  n ,  )  <  d(  n )  +  t . 


I  lie  re  fo  re  , 


inf  .  T  (  n  )  =  inf  .l(rt). 


Now  f'lko  '  n  such  *>»•'**  >'n(x)  =  1  for  |  x  |  <  n,  Un  ( z )  =  minfz.n), 
and  the  co r respond i np  .Jn(n).  To  complete  the  proof  it  suffices  to 
shou  that  "  -M  "  )  uniformly  on  91  as  n  *  »- .  lor  brevity,  we 

Mite  A  f  -  A |  ’ 1 ' .  We  have  from  ( A  ^  ]  ,  5  4, 


(  *  I 


n  2  '  ;i  l  ‘  L  1  '  ,  II  t  •  .  A  (  s  -  HnMnl  (•  ,llt),At  | 


(  1  'nH  1  +  I  x  | 1,1  )  ,  A  t  -  d  n  ♦  j  •  ■  ]♦  I  x|T,1,Ai  -d 


♦ 
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where  Bn  =  {  <  (1  +  |  x  |  m ,  At>  >  C^n}.  Let  £  >  m  as  in  (A£)  and 
p  =  m  £ .  From  Holder's  inequality 


<(l-Pn)(l+|x|m),At>< 


’  ' 

r 

1/p’ 

r 

IdA 

J  t 

i 

,|x|>n 

(l*|xr)PdAt 


]R 


1/P 


<  c  j  n 


,-A/P  ' 


L 

1RN 


(l  +  |  x| *)dA 


l/p+ 1 /p ' 


Since 


P1  ♦  (P '  ) 


and  (p ' )  *  £ 


£  -  m , 


^n<(l-Pn)(l  +  |x|m),At> 


<  c1n'U'm)F7I<i  +  |x| 


•V* 


Ry  Theorem  5.3  the  expectation  on  the  right  side  is  finite.  By 
Cauchy  -  Schwa  r t z 


<l  +  |  x|m,At>dn  <  MBn)1/2[T.T)  <  l  +  |x|m,At>2]1/2. 

n 


Mo  reovc  r , 


11  (pn)  .  c:n  1^n<T  +  I  xlm.At>  . 

Rv  using  again  Theorem  5.3,  the  right  side  of  (.*)  is  hounded  above 
8  1 

!,v  c  >n  •  where  P  =  min(y,f-m).  A  similar  estimate  holds  if 
1  (•,"()  is  replaced  by  (!(•)•  We  then  have,  for  all  n  £  %  t 
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1 


0  <  J(rr)  -  Jn(TT)  <  c2CT+])n‘B, 
as  required.  This  proves  Theorem  6.1. 

Extreme  points  of  21  .  Under  the  hypotheses  of  the  existence  theorem 
4.1  or  of  Theorem  7.2  below,  J(tt)  is  linear  and  lower  semi cont inuous 
on  the  compact,  convex  set  ?1.  Hence,  J(tt)  has  a  mimimum  at  some 
extreme  point  of  21.  Let 

21  =  (extreme  points  of  21}. 


It  can  be  shown  that  ?lS  <=  2Ie  .  However,  the  following  counter¬ 
example,  due  essentially  to  Varadhan,  shows  that  ?(s  f  2le. 

An  example  of  Cirelson  [3]  provides  a  bounded  causal  drift 
coefficient  v(t ,n),  such  that  the  stochastic  differential  equation 


dht  =  a(t,n)dt  +  dY, 

with  Y  a  Wiener  process,  Hq  =  Yq  =  0  has  no  strong  solution. 
However,  the  Carmeron-Martin-Girsanov  formula  gives  a  weak  solution, 
uniquely  determining  the  joint  distribution  measure  n '  of  (Y,n) 
on  C([0,T];R2).  Let  °k  -  [-1,1],  and  U  =  4>  where 

<P  (u)  =  (l-u2)"1u,  -1  <  u  <  1.  Let  4>(Y,U)  =  (Y,n),  nt  =  4>(Ut).  Tlien 


_ 
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- 1  g 

71  =  $  7i'  is  in  21,  but  not  in  21  since  no  strong  solution 
exists.  In  fact,  n  £  21e .  To  see  this,  suppose  that 
tt  =  Xtt  +(1-X)h  ,  0<X<l,n.  £51  for  i  =  1,2.  Let  it!  =  4>7T., 
Then ,  fo  r  i  =  1  ,  2 


if  -  Cn 


t 


•t 

a(s,p)ds  +  Y  ,  0  <  t  <  T)  =  1. 

Jo  t 


Hence,  n!  =  71 '  ,  ik  =  tt  ,  i  =  1,2,  which  implies  n  £  2lc. 

The  following  characterization  [17]  of  2lc  was  pointed  out 

to  the  authors  by  J-M.  Bismut:  11  £  21e  if  and  only  if  every 

.  •? 

bounded  if,  {.^“[-martingale  has  the  form 


Mt  =  c  + 


f  JY 
,  s  s 


with  c  a  constant  and  f^  some  integrablc  predicatable  process 


The  authors  wish  to  thank  V.L.  Penes,  J-M.  Bismut,  and 
S.R.S.  Varadhan  for  helpful  comments  in  connection  with  the  present 
section  6. 
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7.  The  nondegcncratc  case.  Let  us  now  assume,  instead  of  (Aj  ) 
in  §  2 : 


(Aj)  o  is  a  bounded,  continuous  N  x  N  matrix- valued 
function  on  ]R**+**  with  bounded  inverse.  Moreover,  8o/3x^ 

Cb  QRN+M)  for  i  =  1,,..,N. 

We  also  assume: 

(A^)  The  distribution  \i  of  Xq  has  a  density 

p0  £  lzqrn). 

2 

Let  us  show  that,  for  fixed  (Y,U)  £  fi  ,  the  forward 

equation  (5.4)  is  still  correct,  if  suitably  interpreted  in  the 
2 

L  theory  of  parabolic  partial  differential  equations. 

Consider  the  Sobolev  space 


II1  =  {v  £  L2  (IRN)  :  |£-  €  L2QRN),  i  =  1,...,N}, 


- 1  1 

and  II  =  (II  )'.  Let  L  be  the  bounded  linear  operator 
from  H  to  II  *  ,  such  that  for  all  p,  v  £  II* 


/V  !  IN 

<IcP,v>  =  -  2  _  l 


N 


*,1=1  V 


.  .  |E-  p-  dx  +  l 


l j  3x  ^  3xj 


aip  hr  dx 

i=l  1  N  1  9xi 
]R 


+  y  Y 
2  s 


N 

v  f  3  h  3  p  , 

L  aii  aV"  vdx  * 


i ,  j  =  l 


1R 


M  i  j  3  x  .  3  x  . 
N  ■’  j  i 


where  <,>  denotes  here  pairing  of  II*  and  II  *  and 


rr 
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N  9a. 


a^SjX)  =  bi(x,Ys,Us)  -  2  l 


2  . L ,  9x . 
J=1  J 


11  (x,Vs) 


I  Ys'jJ1aij(x'Ys)377 


J 


In  the  "regular  case"  integrations  by  parts  show  that  equation  (5.4) 
is  equivalent  to 


(7.1) 


P (0)  =  pQ,  where 


e(s,x)  =  \  (aYs-Vh,Ys-Vh)  -  b-(Ys*Vh)  -  \  | h | 2 , 


„  .  N  9a.. 

b  i  (s  ,  x)  =  bi(x,Ys,Us)  -  j  I  93c7i^x,Ys')  * 

J=1  J 


The  initial  value  problem  has,  for  fixed  Y,U,  a  unique  solution 

[lj 


p  e  L  2  ( [  o  ,  T  ] ;  H 1 )  n  c([0,T]  ;L2QRN))  . 

Theorem  7.1.  q(t)  =  p (t)exp (Yt -h)  is  the  density  of  the  unnormalized 
conditional  distribution  A  . 

1  Proof.  From  (5.6),  this  is  true  in  the  regular  case.  Following 

the  proof  of  Theorem  5.1,  we  make  approximations  °n’*3n ’^n ’^n  ’  sucb 
that  °n  ’  9°n/|,8xi  ’  ^n  ’  ’  ^n  are  un^ormlY  bounded  and  tend 

uniformly  to  o,  9o/  9X  .  , .  .  .  ,h  as  n  00  with 

i  *  1 

an  -  °n  °n  >  uI(a  >  0)  for  all  n.  is  continuous  and  tends 

?  00  M 

to  U  strongly  in  L  ( [0  ,T]  ;  <52r)  ,  while  u  has  density  p  OR  ) 

n  '  no  0  ' 

,  ■  2  w 

tending  to  pQ  strongly  in  L  (IR  )  .  The  density  p  (t)  of  the 
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corresponding  Ant  satisfies 


dp 

-r-r-  -  L  4.V  +  e  p„ 

dt  ntrn  n*n 

p  (0)  =  p  , 

n  *no 

are  obtained  by  replacing  above  by 

A  A  A  A 

Rewrite  Ap  =Lp  +ep,  and  Ap  =  Lp  +  ep.  Then: 
n1  n  nrn  n1  n  1  1  1 

<  3t  *  Vp-prP  *  Sn 

P(0)  -P„(0)  -  P„  ■  p„0 

V. 

where  g  =  (A -A  )p. 
bn  n'1 

It  follows  from  the  above  hypotheses  that  there  exists  c, 
independent  of  n,  such  that  for  all  v  E  H  : 


where  L  .  ,  e 
nt  n 


o  , .  .  .  ,U  .  . 
n  ’  ’  n 


<-A  v,v>  +  c | v | 2 2  N  >  \ 
n  Lz0RN)  z 


,1- 


Consequently,  by  standard  PDE  arguments,  see  [1],  there  exist 
'  and  c"  such  that 


sup  | p(t)  -  p  (t) |  2  N  <  c' | p 

0<t<T  n  V(3RN) 


nolL20RN) 


*  C  L2C0,T;H-b 


One  easily  checks  that  g^  -*■  0  in 
Finally,  pn(t)  -+■  p(t)  in  L 2  (3RN) . 


L2(0,T;H'1). 

Then 


lim  <f,A  >  =  lim 

nt 

n-x»  n-*30 


f  exp(Yt-h)pndx  =  f  exp(Yt-h)pdx 


]R 


N 


TK 


N 


,N, 


for  any  f  €  CqQR  ).  However,  the  proof  of  Theorem  5.1  showed 
that  <f,A  t>  -*■  <f,At>.  Thus,  q  =  p  expfY^-h)  is  the  density  of 
A  ,  which  is  Theorem  7.1. 

Y  U 

Let  us  write  p  =  p  ’  to  emphasize  the  dependence  on  Y,U 
of  the  solution  to  the  initial  value  problem  (7.1).  From  (4.2)  and 
Theorem  7.1  we  can  rewrite  the  criterion  to  be  minimized  as 


(7.2)  J(tt) 


[  j  j  F(x,Ut)  pY,U(t,x)exp(Yt-h(x))dxdt 

.2  J°  V 


+  I  G(x)pY,U(T,x)exp(Y  •h(x))dx]d7T(Y,U) . 

N  1 

m 


Let  us  suppose: 

(A '^)  Condition  (A,-)  in  §4  holds,  and  F(x,‘)  is  convex 
on  for  all  x  £  1RN. 

Theorem  7.2.  There  exists  n*  £  8  such  that  Jt77*)  <  Jf77)  for 
all  77  €  ?1. 


Let  us  first  prove  two  lemmas. 


Lemma  7.1.  For  every  P  £  C^QR^),  p  >  0,  and  (Y,U)  £  the 
function  ^  (V)  defined  by 


[  (  P(x)F(x,V  )pY»U(t,x)exp[V. -h(x)]dxdt 

J  nJ  N  r 

u  ^ 


*(V) 


is  lower-semicontinuous  on  • 

Proof:  Since  (V)  is  convex  from  (A£),  it  suffices  to  show 

,  -  .  .  2 

that  it  is  continuous  on  L  (0,T;*2O  endowed  with  the  strong 
topology. 

Let  Vn  V  in  L. 2  ( 0 , T ;  strongly.  Let  Vn  be 

a  subsequence  such  that  converges  for  almost  all  t.  Then 

n  * 

*CV  )  •*  P  (V)  .  Consequently,  any  convergent  subsequence  of 
(>KVn)}  has  <|»(V)  as  its  limit.  But  V  is  uniformly  bounded 
on  L2(0,T;^).  It  follows  that  ^  (Vn)  -*■  <p  (V)  . 

Lemma  7.2.  Let  (Yn,Un)  ->  (Y,U)  in  ft2.  Denote  pn  = 

Yn’Un  v  „ 

P  ,  p  =  p  » u  Then  for  every  D  bounded  open  subset  of 

N 

1R  with  smooth  boundary, 

(a)  pn(T)  -*•  p  (T)  in  L2(D)  weakly. 

(b)  pn  ->  p  in  L2((0,T)  x  D)  strongly. 


Proof •  Equation  (7.1)  can  be  rewritten  in  the  form: 


<A„p,v> 


1  V  f  IP  3v  ,  r  3v 

-  I  i  j.!  J  aij  3XS  Sir  Ox  j/V  377 

TT»  *  ’ 


N  f: 

£  V.  v  dx  +  6pv  dx. 

i  =  l  JmN  dxi  LN 


<A1p,v>  = 


N  f  * 

*  N^ip  1x7  dx 
1  =  1  LN  l 


+  0pv  dx 

J  N 

m 


with 


N  da-  . 


aiCt>x)  =  -hi(x>Yt)  +  i  arr1  (x»Yt)  +  I  Yt-^1aij(x’Yt)^T 


9h(x) 


YjCt.x)  = 


6(t,x)  = 


bi(x’Yt) 


2  Yt‘j^aij(x’Yt)  dIJ  M 

■7  CaYt-Vh,Yt-Vh)  *  Yt-  (x)[b“(x,Yt)  - 

-  7  H71  <x.Yt)]  ■  \  |h|2(x) 


«(t,x)  -  Yt-.I^Itx.Yj)  (x) 


all  these  coefficients  being  continuous  and  bounded  functions  of 


(t,Yt)  . 


It  follows  from  standard  arguments,  after  multiplication  of 


(7.3)  by  p  and  making  use  of  (A£) ,  that  there  exists  a  unique 

constant  K  (depending  only  on  sup|Y  |)  such  that: 

t  z 
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(7.4)  |p(t)|Z  N 

LZORN) 


I  v p  C s }  I  2  jy[^s  5  I  Pq  I 
o  (i/oRN)r 


+  K 


f  |P(s)| 
J  n 


L20RN] 


Let  now  (Yn,Un)  "*■  CY , U)  in  fi2.  Then  sup  is  uniformly 

bounded,  and  it  follows  from  (7.4)  and  (7.3)  that  (p n  ,pn  (T) ) 

remains  in  a  bounded  subset  of  L2(0,T;H^)  x  L2(0,T;H  x  L20R^). 

We  can  then  extract  a  subsequence,  still  denoted  Pn ,  such  that: 

.  n 

(pn ,  ,  pn  (T) )  -*■  (P,q,r)  weakly,  and  it  is  easy  to  check  that 

q  =  ^  ,  r  =  p(T)  .  Then  p  £  L2  (0  .TjH'*’)  .  If  we  still  denote  by  pn 
and  p  the  restriction  of  pn  and  p  to  [0,T]  x  D,  we  have: 

(i)  pn  -*■  p  in  L2 (0 ,T ;H^ (D) )  weakly 

(ii)  in  L2(0,T;H‘1(D))  weakly 

(iii)  pn(T)  p(T)  in  L2(D)  weakly 

where  D  is  open,  bounded  and  with  smooth  boundary. 

Since  D  is  bounded,  the  injection  from  H  (D)  into  L^(D) 
is  compact,  and  it  follows  from  (i)  and  (ii)  by  a  compactness  Lemma 
[11]  that 

(iv)  pn  -*■  p  in  L2([0,T]  x  D)  strongly. 

—  00  _  1 

It  remains  to  show  that  p  =  p.  Choose  any  4>  £  CqQR  anc* 
v  £  Cq(D).  Multiply  (7.3)n  by  4>v,  and  integrate  by  parts: 


ds. 


+ 


4>(T)(pn(T)  ,v) 


+  [  «j>  ( t )  <p 
J  Q 


n 

9 


(Aq) *v>  dt 


15 


+ 


+ 


1 


T 

0 

T 

0 


u"Ht)<pn,  (Aj)*v>dt=  4>(0)(p0,v) 
&  (pn»v)dt 


2  N 

where  (  ,  )  denotes  the  inner  product  in  LZQR  ). 

Now,  (Aq)  v->-AqV  in  L^(0,T;H  *(D))  strongly  and 
(A^)  *v A*v  in  L2([0,T]  x  D)  strongly.  It  follows  from  (i)  , 

(iii)  and  (iv)  that  we  can  take  the  limit  in  the  above  equality. 

Since  D,v  and  $  are  arbitrary,  p  =  p,  the  unique  solution  of  (7.3). 


Proof  of  Theorem  7.2.  As  in  Theorem  4.1,  it  suffices  to  show  that 
J(tt)  is  lower  semi -continuous  on  21,  and  this  will  be  true  if 
for  all  P  €  Co0RN),  P  >  0,  and  H  €  C^QR1)  montone,  the  following 
functional  is  lower  semi -continuous  on  21: 


JO1) 


P  (x)  F  (x  ,Ut )  exp  (Yt '  h  (x)  )pY  ,U 


(t,x)dxdt 


+ 


Y,IJ 


P(x)G(x)exp  (Yj'h(x))p  (T,x)dx 


K 


d"(Y,U). 


A  sufficient  condition  for  J  to  be  l.s.c.  (lower  semi -continuous)  on  21 

2 

is  that  the  integrand  be  l.s.c.  on  ft  . 

Since  H  is  continuous  and  monotone,  it  suffices  to  show  that 

2 

the  following  functional  is  l.s.c.  on  ft  : 


f 
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Tf  Y  ,, 

dDCY.U)  =  I  N  P  (x)I;(x,U  )exp(T  -h(x))p  ’  (t,x)dxdt  + 

J0J]R  L 

+  (x)exP  (Yj'h(x  )  )pY,U(T,x)dx  . 

Let  now  (Yn,Un)  be  a  sequence  such  that  (Yn,Un)  •+  (Y,U) 

2 

in  ft  ,  and  consider  (with  the  notations  of  Lemma  7.2): 

T 

(H)(Yn,Un)  -  0(7,U)  =  f  NP[F(Un)  -  F  (U)  ]  exp  (Y.-h)p  dxdt  + 

'0J1R  x 

T 

+  WPF  (Un)  [exp  (Y^-h)pn  -  exp  (Y  -h)p]  dxdt  + 

J0JIR  z 

+  mPG  [exp  (Y~-h)pn  (T)  -  exp  (YT*h)P  (T)  ]  dx . 

JJR  1  1 

When  n  ,  it  follows  from  Lemma  7.1  that  lim  inf  of  the  first 
term  in  the  right  hand  side  is  >0.  The  two  other  terms  tend  to  zero 
from  Lemma  7.2.  Then  (0) (Yn,Un)  >  (U)(Y,U). 


-r 


[  appi;ndix 

^  In  this  Appendix  we  prove  two  results  used  in  the  paper.  The 

first  result  concerns  the  continuous  dependence  on  the  coefficients 
and  initial  state  of  solutions  to  martingale  problems  associated 

1 

with  stochastic  differential  equations  of  the  form 


(A . 1 )  dXt  =  (p°(t,Xt)  +  B1 (t,Xt)Ut)dt  +  Y(t,Xt)dWt,  0  <  t  <  T 


X0  =  x. 


Let  us  write  for  brevity  =  (W  ,Xt),  and  consider  the  "canonical" 

sample  space  in  the  notation  of  §2.  l  or  f  6  CqQR^  let 


ft 


L;f(x’)ds, 


(A. 2)  Mf(t)  =  f(X’)  -  f(X^)  - 

'  V 

(A. 3)  L’C  ■  iAwf  .  ‘  l 

1  »  J  ~  * 


N  11 

*  T.  ,i1Yik(t’x,fx.ul 

i  j  1=1  K= 1  l  k 


(g°(t,x)  +  e1(t,x)ut)-vxf 


:  .  where  A  f(w,x)  is  the  Laplacean  with  respect  to  w,V  the  gradient 

»  W  X 

in  x,  and  a  =  yy'.  The  martingale  problem  is  to  find  a  probability 
measure  Px  on  ( )  such  that  Px(Xq  =  =  *  anc^  Mf(t)  is  a 

Px>  martingale  for  every  f  £  CqQRD  +  ^).  See  [14,  Ch.  6]. 

Let  us  call  a  function  g  of  class  y^  if  B  is  Borel 

‘  A  M 

measurable  on  [0,T]  *  1R  ,  |g(t,x)j  <  K,  and  g(t,-)  is  Lipschitz 
1  ,  with  constant  K.  If  g^,g*,Y  are  of  class  yv  and  U  e  ( [0  ,T]  ;  )  , 

I  ‘  then  the  Ito  conditions  hold  in  (A.l).  This  implies  existence  and 


uniqueness  pathwise  of  solution*  to  (  A .  1  )  , 


Is 

and  consequently  existence 

and  uniqueness  of  the  solution  P  to  the  in;irt  liusile  problem. 

We  write  I’  for  the  solution  to  the  martinrale  problem 

11  x  1 

if  t^'.i.U  arc  replaced  bv  ff,>  ,U  ,  n  =  1  ,  »'  -  0 ,  1  . 

i  n  n  n 

r 

Lemma  A.l.  Assume  that  P'  ,  i  are  of  class  and  tend  to  f‘ 

as  n  -*■  *  ,  uniformly  on  compact  subsets  of  ( 0 ,  I  |  •  F '' ,  •  =  0  ,  1  . 

a 

Moreover,  assume  that  ►  U  weakly  in  I  " ( ( 0  ,  i  ) ;  & ) ,  x  »  x  as 

n  *  r>  .  Then  I'  -»  I' 

-  nx  x 

n 

Proof.  The  sequence  I1  =  P  of  prohabilitv  measures  is  tight 

-  n  n  x  ™ 

n 

j  11 ,  (  h .  n.l].  Hence  ,  am  subsequence  has  ,j  t  ur !  h<  i  ••  u!’St  ,  urn i  e  t  en  !  i  m: 

to  a  limit  P^.  It  suffices  to  show  that  P  ^  is  a  solution  to  the 

martingale  problem.  Uniqueness  then  implies  P()  =  P  .  Clearly, 

Pn ( X '  =  ( 0  ,  x ) )  =  1 .  he  t  us  write  M  r , 1 '  in  I  A . 2  )  ,  ; A . 3 1  when 
00  nf  nt 

9.  ? 

6  ,>,U  are  replaced  by  *?n*\i’l,n'  *et  us  s*'ow  L  h  a  t  for  fixed 
f  €  ('~0r"  ^ )  and  compact  F  c  , 

(A. 41  1 1  m  1.’  f(X’)ds  =  1. '  f  (X  '  )  ds 

n-«  J0  n?  s  Jo  s 

uniformly  on  [0,T]  *  F.  Since  Mnj.(t)  is  a  l’n ,  {  martingale 

and  P^  ■*  P(1  [n  in  a  subsequence)  (A.  4)  will  imply  that  M^-ft) 
is  a  P(1 ,  f  }  martingale.  Now 


Lnd  -  Kf  ’  ns'',f  *  *  %<*.*>, 


where  0n  -*■  0  uniformly  on  compact  subsets  of  [0,T]  x  ip^,  Since 
^ns  *s  bounded  (see  (A^),  §2)  and  ->  g^  uniformly  on 


i  OH  1 1 


un  i  r 
s  how 

i  \.  5 

un  i  t' 
!  IP' 
un  l  t 
i  n  t  e 
ro<p 

i  \ .  i 

vs  ho  l 

oho 

Vs  o  1 1 

Soto 
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,io  t  u'is 

1  im  ^  [  t'Vs  )  -  f?1  (s  ,X  )  ]U  -v  r  (  X  ^ )  o!  s  =  0 
l . 1  n  s  s  ns  x  s 

n  1  0 


o  nn  1  v  lor  0  '  t  •  1  ,  \  £  1.  To  obtain  [A. 4)  it  remains  to 

t  ha  t 


ix  i 

)  1  in.  1  ‘is  .xs  i m  -tij  -Vf(x;)as  =  0 

n  M) 

ormlv  on  [0,'I|  '  Now  b*(s,-'|  and  Vf  are  bounded  and 

obit;,  with  some  constant  K.  Moreover,  functions  X'  €  T  are 
ormlv  homulosl  ami  cqu  i  con  t  i  minus  .  Therefore,  given  o  >  0  the 
oral  in  l  \ .  1)  can  be  approximated  to  within  1 ,  uniformly  with 
oct  t 'i  \*  t  :  and  n  =  1,2,...,  by  a  finite  sum 

r 

r  1  i 

'  ■  v  i  s  ,X  .  )  (II  II  I  f  (  X.’hls 

I  i  ,  i  n  s  s  i 

1  1  1 

o  0  -  t  t  t  =  I  and  x x '  arc  suitably 

v'l  m  1  m 

on.  i  I  ho  t  ,x'  depend  on  t.  and  f.  1  Since  II  -*  U 
i  i  n 

lv,  it.'  1  tends  to  0  as  n  »  «’ .  Ill  i  s  proves  Lemma  A.l. 

In  tliis  paper  we  appeal  to  I.emma  A.l  three  times.  In 


I  emna  2.1  we  take 
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Yn(t,x)  =  o(x,Ynt),  Y(t,x)  =  o(t,Yt) 
g^(t,x)  =  b^  (x  > Ynt ) ,  6£(t,x)  =  b£(x,Yt) 


where  |  |  Y  -  Y  j  |  0  as  n  -*■  00  (sup  norm).  In  that  case, 

YU  £ 

Px  =  Px’  .  In  the  proof  of  Lemma  3.1  we  use  instead  of  b 

V  £ 

the  modified  drift  coefficient  b  corresponding  to  the  change  of 

VY  U 

probability  measures  (3.4).  Then  Pxr  =  P  ’  and  L<-  is  replaced 

XX  3 

V 

by  Ls  in  (3.5).  Finally,  we  use  Lemma  A.  1  in  the  proof  of 
Theorem  5.1  as  indicated  there. 

In  §6  we  postponed  the  proof  of  Lemma  6.2. 


Proof  of  Lemma  6.2.  Since  8  c  a  it  suffices  to  show  that, 
-  m  m 

for  every  71  €  ?I  and  e  >  0,  there  exists  n  e  ?fS  such  that 
'  m  1  m 


► 

Yd",  <  Ip dTT  +  e, 

’  1  J  7 


Let  us  fix  A  and  consder  different  T  =  mA,  m  =  1,2,...  .  IVe  prove 
(#)  by  induction  on  m.  For  m  =  1,  each  admissible  control  n , 
with  U  constant  on  [0,A)  r-almost  surely,  corresponds  to  a 
product  measure:  n  =  w  *  a  where  w  is  Wiener  measure  on 
=  C([0,A];  IR^)  and  <*  is  a  probability  measure  on  <2r. 


Let  u  minimize 


V (Y  ,u) dw (Y)  on  The  control  such 


that  Ut=u,0<t<A  with  probability  1  satisfies 


nY,u  )dw(Y), 


♦  <*' 


where  12^ 


51 


=  S6 


x 


21 


fi,  , 


We  then  have 


d  TT  = 


(Y  ,  u)  dw(Y)  d«  (u)  > 


^  dTT  , 


°k  n  7 


l 


as  required. 

Now  suppose  that  (#)  has  been  proved  when  in  is  replaced  by 

m  -  1  (i.e.,  T  by  T  -  A).  Let  (Y,U)  denote  the  restriction  of 

(Y,U)  to  [ 0 , T  -  A ] ,  and  the  space  of  such  (Y,U).  Let  n  be 

_2 

the  measure  on  induced  from  n  by  restriction.  We  write 

_ 7 _ 

similarly  when  T  is  replaced  by  T  -  A.  Let 

H  =  U  on  [T-A ,T)  ,  where  U  6  Let  Y.=Y  -  Y  ~  .  on 

[T-A,T],  and  w  Wiener  measure  on  C  C ( [T-A , T 1 .  We  can 
L  m  m 

identify  Y  with  (Y,Y  )  and  piecewise  constant  U  with  (U,Um). 
Let 


Y  ( Y  ,  U  ,  u ) 
;(Y,U)  = 


=  <P  (Y  ,Y  ,U,u)dw  (Y  ), 
J  ^  v’m  m  nr  ’ 

m 

min  Y ( Y , U , u ) . 

% 


? 

Since  4>  is  bounded  and  continuous  on  any  compact  subset  of  fi  , 

Q  is  bounded  and  continuous  on  any  compact  subset  of 

Consider  any  n  £  $1  ,  with  corresponding  tt  determined  bv 

restriction.  By  induction  there  exists  "j  strict-sensc 
admissible,  such  that  Ut  is  constant  on  [t.  j,t.),  j  =  l,...,m-l, 
11  j -a  I  most  surely,  and 


•  __ 
4dTT1 


<  j  C  d  *t  + 

_ 7 

n 


e 

3  ' 


—2  —2 
we  define  <p  :  ft  -*■  °k  as  follow?.  Let  K  cz  ft  be  compact  with 


where 


M«Z-K)  +  wi  CQ2-K)  <  eC3|  M  I)'1, 


sup  norm.  Choose  a  partition  K  =  U  . . .  U 


with  each  K.  6  .  and  (Y.,U.)  £  K.  such  that 

l  1-A  li  l 


;(Y,,U.)  <  C(Y,U)  +  -  ,  Y ( Y , U , u )  <  Y ( Y - , U • , u )  + 

XI  n  d  J. 


for  all  (Y,U)  £  K.  ,  u  £  ^ .  Let  u  £  minimize  Y(Y-  ,U-  ,u)  on  , 

l  j  ii 

i  =  l,...,n.  Let  Uq  £  ^  be  arbitrary;  and  take 


<f>(Y,U)  = 


Uj,  (Y,u)  e  K. 


un,  (Y , U)  €  n  -  K. 


The  control  is  defined  by  taking  =  <}>(Y,U)  n^-almost 

surely,  and  the  restriction  of  77  ^ .  Then 


Y(Y,U,«l>(Y,U))dn1 


<  l  \  ^(Yi,Ui)d7ri  +  j  Y(Y,U,u0)dni  ♦  , 

1  1  IV  .  r\  ~  1/ 


-  l_2Cd?l  *  ¥  S  ’  ; 

ST 


ft  -k 


Cd77  +  e, 


On  the  other  hand. 


■  *>  « 


This  gives  (#),  and  hence  Lemma  6.2. 
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